From scans to models: Registration of 3D human shapes exploiting texture information

New scanning technologies are increasing the importance of 3D mesh data, and of algorithms that can reliably register meshes obtained from multiple scans. Surface registration is important e.g. for building full 3D models from partial scans, identifying and tracking objects in a 3D scene, creating statistical shape models. Human body registration is particularly important for many applications, ranging from biomedicine and robotics to the production of movies and video games; but obtaining accurate and reliable registrations is challenging, given the articulated, non-rigidly deformable structure of the human body. In this thesis, we tackle the problem of 3D human body registration. We start by analyzing the current state of the art, and find that: a) most registration techniques rely only on geometric information, which is ambiguous on flat surface areas; b) there is a lack of adequate datasets and benchmarks in the field. We address both issues. Our contribution is threefold. First, we present a model-based registration technique for human meshes that combines geometry and surface texture information to provide highly accurate mesh-to-mesh correspondences. Our approach estimates scene lighting and surface albedo, and uses the albedo to construct a high-resolution textured 3D body model that is brought into registration with multi-camera image data using a robust matching term. Second, by leveraging our technique, we present FAUST (Fine Alignment Using Scan Texture), a novel dataset collecting 300 high-resolution scans of 10 people in a wide range of poses. FAUST is the first dataset providing both real scans and automatically computed, reliable ground-truth correspondences between them. Third, we explore possible uses of our approach in dermatology. By combining our registration technique with a melanocytic lesion segmentation algorithm, we propose a system that automatically detects new or evolving lesions over almost the entire body surface, thus helping dermatologists identify potential melanomas. We conclude this thesis investigating the benefits of using texture information to establish frame-to-frame correspondences in dynamic monocular sequences captured with consumer depth cameras. We outline a novel approach to reconstruct realistic body shape and appearance models from dynamic human performances, and show preliminary results on challenging sequences captured with a Kinect

psort

psort \ue8 stato il pi\uf9 veloce software di ordinamento per macchine di classe PC dal 2008 al 2011 (benchmark Pennysort, http://sortbenchmark.org) e un suo adattamento per cluster ha migliorato il record per il benchmark datamation di quasi un ordine di grandezza nel 2011. Il rapporto tecnico ufficiale si trova sul sito sortbenchmark.org (che cataloga i pi\uf9 efficienti software di ordinamento per varie categorie di task/hardware - originariamente mantenuto dal premio Turing Jim Gray) all'URL http://sortbenchmark.org/psort_2011.pdf -- Ulteriori dettagli si possono trovare nelle pubblicazioni: P. Bertasi, M. Bressan, E. Peserico. psort, yet another fast stable sorting software, ACM Journal of Experimental Algorithmics, vol. 16, 2011 -- P. Bertasi, M. Bonazza, M. Bressan, E. Peserico. Datamation: a quarter of a century and four orders of magnitude later. Proc. of IEEE CLUSTER 201

Spatio-Temporal Human Shape Completion With Implicit Function Networks

International audienceWe address the problem of inferring a human shape from partial observations, such as depth images, in temporal sequences. Deep Neural Networks (DNN) have been shown successful to estimate detailed shapes on a frame-by-frame basis but consider yet little or no temporal information over frame sequences for detailed shape estimation. Recently, networks that implicitly encode shape occupancy using MLP layers have shown very promising results for such single-frame shape inference, with the advantage of reducing the dimensionality of the problem and providing continuously encoded results. In this work we propose to generalize implicit encoding to spatio-temporal shape inference with spatio-temporal implicit function networks or STIF-Nets, where temporal redundancy and continuity is expected to improve the shape and motion quality. To validate these added benefits, we collect and train with motion data from CAPE for dressed humans, and DFAUST for body shapes with no clothing. We show our model's ability to estimate shapes for a set of input frames, and interpolate between them. Our results show that our method outperforms existing state of the art methods, in particular the single-frame methods for detailed shape estimation

Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction

International audienceModeling hand-object manipulations is essential for understanding how humans interact with their environment. While of practical importance, estimating the pose of hands and objects during interactions is challenging due to the large mutual occlusions that occur during manipulation. Recent efforts have been directed towards fully-supervised methods that require large amounts of labeled training samples. Collecting 3D ground-truth data for hand-object interactions, however, is costly, tedious, and error-prone. To overcome this challenge we present a method to leverage photometric consistency across time when annotations are only available for a sparse subset of frames in a video. Our model is trained end-to-end on color images to jointly reconstruct hands and objects in 3D by inferring their poses. Given our estimated reconstructions, we differentiably render the optical flow between pairs of adjacent images and use it within the network to warp one frame to another. We then apply a self-supervised photometric loss that relies on the visual consistency between nearby images. We achieve state-of-the-art results on 3D hand-object reconstruction benchmarks and demonstrate that our approach allows us to improve the pose estimation accuracy by leveraging information from neighboring frames in low-data regimes

From scans to models: Registration of 3D human shapes exploiting texture information

New scanning technologies are increasing the importance of 3D mesh data, and of algorithms that can reliably register meshes obtained from multiple scans. Surface registration is important e.g. for building full 3D models from partial scans, identifying and tracking objects in a 3D scene, creating statistical shape models. Human body registration is particularly important for many applications, ranging from biomedicine and robotics to the production of movies and video games; but obtaining accurate and reliable registrations is challenging, given the articulated, non-rigidly deformable structure of the human body. In this thesis, we tackle the problem of 3D human body registration. We start by analyzing the current state of the art, and find that: a) most registration techniques rely only on geometric information, which is ambiguous on flat surface areas; b) there is a lack of adequate datasets and benchmarks in the field. We address both issues. Our contribution is threefold. First, we present a model-based registration technique for human meshes that combines geometry and surface texture information to provide highly accurate mesh-to-mesh correspondences. Our approach estimates scene lighting and surface albedo, and uses the albedo to construct a high-resolution textured 3D body model that is brought into registration with multi-camera image data using a robust matching term. Second, by leveraging our technique, we present FAUST (Fine Alignment Using Scan Texture), a novel dataset collecting 300 high-resolution scans of 10 people in a wide range of poses. FAUST is the first dataset providing both real scans and automatically computed, reliable ground-truth correspondences between them. Third, we explore possible uses of our approach in dermatology. By combining our registration technique with a melanocytic lesion segmentation algorithm, we propose a system that automatically detects new or evolving lesions over almost the entire body surface, thus helping dermatologists identify potential melanomas. We conclude this thesis investigating the benefits of using texture information to establish frame-to-frame correspondences in dynamic monocular sequences captured with consumer depth cameras. We outline a novel approach to reconstruct realistic body shape and appearance models from dynamic human performances, and show preliminary results on challenging sequences captured with a Kinect.Lo sviluppo di nuove tecnologie di scansione sta accrescendo l'importanza dei dati tridimensionali (3D), e la necessita' di algoritmi di registrazione adeguati per essi. Registrare accuratamente superfici 3D e' importante per identificare oggetti ed effettuarne il tracking, costruire modelli completi a partire da scansioni parziali, creare modelli statistici. La registrazione di scansioni 3D del corpo umano e' fondamentale in molte applicazioni, dal campo biomedico a quello della produzione di film e videogiochi; ottenere registrazioni accurate e affidabili e' pero' difficile, poiche' il corpo umano e' articolato, e si deforma in maniera non rigida. In questa tesi, affrontiamo il problema della registrazione di scansioni 3D del corpo umano. Iniziamo la nostra analisi considerando lo stato dell'arte, e rilevando che: a) la maggior parte delle tecniche di registrazione 3D usa solo informazione geometrica, che e' ambigua in zone in cui le superfici sono lisce; b) c'e' una mancanza di adeguati dataset e benchmark nel settore. L'obiettivo di questa tesi e' quello di risolvere questi problemi. In particolare, portiamo tre contributi. Primo, proponiamo una nuova tecnica di registrazione per scansioni 3D del corpo umano che integra informazione geometrica con informazione cromatica di superficie. La nostra tecnica dapprima stima l'illuminazione nella scena, in modo da fattorizzare il colore della superficie osservata in effetti di luce e pura albedo; l'albedo estratta viene quindi usata per creare un modello 3D del corpo ad alta risoluzione. Tale modello viene allineato a una serie di immagini 2D, acquisite simultaneamente alle scansioni 3D, usando una funzione di matching robusta. Secondo, sulla base delle registrazioni prodotte dalla nostra tecnica, proponiamo un nuovo dataset per algoritmi di registrazione 3D, FAUST (Fine Alignment Using Scan Texture). FAUST colleziona 300 scansioni 3D relative a 10 soggetti in differenti pose. E' il primo dataset che fornisce sia scansioni reali, sia registrazioni accurate e affidabili ("ground truth") per esse. Terzo, esploriamo possibili usi del nostro approccio in dermatologia. Combinando la nostra tecnica di registrazione con un algoritmo di segmentazione per lesioni melanocitiche, proponiamo un sistema di screening in grado di rilevare l'insorgenza di nuove lesioni o modifiche in lesioni preesistenti su quasi tutta la superficie cutanea; tale sistema e' di aiuto per i dermatologi nell'individuazione di potenziali melanomi. Concludiamo questa tesi esaminando l'importanza di usare informazione cromatica per registrare scansioni 3D acquisite in sequenze dinamiche. In particolare, proponiamo un nuovo approccio per ottenere modelli 3D realistici e completi del corpo umano a partire da sequenze acquisite con un singolo Kinect

Routing and caching on DHTS

L'obiettivo della tesi e' quello di analizzare i principali meccanismi di caching e routing implementati oggigiorno nelle DHT piu' utilizzate. In particolare, la nostra analisi mostra come tali meccanismi siano sostanzialmente inefficaci nel garantire un adeguato load balancing tra i peers; le principali cause di questo fenomeno sono individuate nella struttura, eccessivamente rigida, adottata dalle DHT e nella mancanza di correlazione tra meccanismi di routing e di caching. Viene quindi proposto un diverso overlay, organizzato in base a una struttura ipercubica, che permetta di adottare un algoritmo di routing piu' flessibile e di sviluppare due meccanismi di caching e routing strettamente interconnessi. In particolare, l'overlay ottenuto riesce a garantire che ogni nodo subisca un carico al piu' costante, con una taglia di cache costante e una complessita' di routing polilogaritmica nel caso peggior

Optimal throughput and delay in delay-tolerant networks with ballistic mobility

This work studies delay and throughput achievable in delay-tolerant networks with ballistic mobility -- informally, when the average distance a node travels before changing direction does not become vanishingly small as the number of nodes in the deployment area grows. Ballistic mobility is a simple condition satisfied by a large number of well-studied mobility models, including the i.i.d. model, the random waypoint model, the uniform mobility model and Levy walks with exponent less than 1. Our contribution is twofold. First, we show that, under some very mild and natural hypotheses satisfied by all models in the literature, ballistic mobility is strictly necessary to achieve simultaneously, as the number of nodes grows, a) per-node throughput that does not become vanishingly small and b) communication delay that does not become infinitely large. Any network whose nodes exhibit a more "local" mobility pattern (e.g. Levy walks with exponent greater than 1, or Brownian motion) must sacrifice either a) or b), regardless of the communication scheme adopted -- even with network coding. Second, we present a novel packet routing scheme. Our scheme is relatively simple and does not rely on centralized control, replication, or static base stations. At the same time it achieves both non-vanishing throughput and bounded delay as the number of nodes grows, on any network with ballistic mobility (i.e. whenever they can be simultaneously achieved), asymptotically outperforming any existing communication scheme that exploits node mobility to boost throughput